Doing Practical Data Science

Author: Robert Chang @_rchang

Overview

A little bit about myself

Academically…

Professionally…

What is a Data Scientist?

There are many open definitions:

My Favorite Definition

images/ds_def.png
Credit: Josh Wills

The Most Famous Venn Diagram in Data Science

images/venn.png
Credit:Drew Conway

What Do Data Scientists Do Exactly?

images/ds_triangle.png
Credit:Thomas Nguyen

Defining Key Performance Metrics

All startups and companies want to make money, become cash positive, and see millions of users using their services. We need to measure against ourselves

Data Warehousing

images/twitter_data_flow.png
Credit:Twitter Data Platform Team

Data Analysis

images/jira_data_analysis.png
Credit: Growth Data Science Team

Hard Core Data Science

images/jira_data_science.png
Credit: Growth Data Science Team

Data Analysis v.s. Data Science, Go!

Data Analysis: An Example

Answering the Question at hand

How many users churned in each country on 03/01/2015?

select
    country_id, count(distinct user_id)
from
(
SELECT
        '2015-03-01' AS created_date, user_id, country_id, deactivated
        FROM growth_staging.full_compact_snapshot_staging
        WHERE created_date = '2015-03-01'
              AND NOT today_mau
              AND yday_mau
              AND NOT suspended
) churned_mau_daily
where created_date = '2015-03-01' and deactivated = 0
group by country_id
order by country_id;

Ok, but this only answered one question, not very efficient

Answering the next N questions

images/vertica_tables.png
Tables created over time

Some thoughts on Data Analysis

Moving on to Data Science…

Data Science: An Example

Machine Learning is Founded in Math

images/ml_1.png
Credit: Jimmy Lin & Dimitriy Ryaboy

Machine Learning != R/Python script

images/ml_2.png
Credit: Jimmy Lin & Dimitriy Ryaboy

Machine Learning is Expressed in Codes

images/ml_3.png
Credit: Jimmy Lin & Dimitriy Ryaboy

Bias to Productionization

images/ml_4.png
Credit: Jimmy Lin & Dimitriy Ryaboy

ML fits well with the Map-Reduce Paradigm

images/ml_5.png
Credit: Jimmy Lin & Dimitriy Ryaboy

How everything fits together

images/ml_6.png
Credit: Jimmy Lin & Dimitriy Ryaboy

Some Thoughts on Data Science

Experimentation & A/B Tests

Experimentation & A/B Tests

images/ddg.png
An example of Experiment Brief

Experimentation & A/B Tests

images/ddg_ui.png
An example of our A/B Tests Framework

Recap: What do data scientists do?

Building a Data Science Team

Recruiting

Thank you!

/

#